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(57) Abstract: System for establishment of a virtual private network connection, comprising an end user client device (1 ) and a 
VPN access server (2) communicatively connected to the end user client via the Internet (4). The system is characterised in that 
it includes a standalone VPN client (3) device physically interconnecting (II, 12, 13) the end user client with the Internet, said 
VPN client comprising monitoring means for monitoring all traffic between the end user client and the VPN server. Preferably said 
monitoring means are devised to detect when a handshake agreement is established between the end user client and the VPN server, 
and to overtake a VPN setup session for said end user client upon detection of said handshake agreement. 
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Field of the invp.nrinn 

The invention relates in general to computer networks, and in particular to 
systems and methods for in customer premises equipment based network access 
servers for secure, dynamic, and fault tolerant establishment of server controlled 
Internet Protocol virtual private networks. 

Background 

Most enterprises are located at multiple sites where each site has its own local 
area network (LAN). A site is defined as anything from a head-quarter, or an 
affiliation company site, to a single employee's remote office site. Some kind of 
communication infrastructure is then used to interconnect the different sites The 
Internet evolution can roughly be categorised into two main areas: 

a) Internet as the global communication infrastructure. Traditionally 
companies used so called leased lines, provided by telephone companies to 
interconnect their sites. Separated firewall solutions were used for accessing the 

20 Internet. During the last years, companies are no longer using Internet only for 
external communication, more and more companies are trying out new network 
solutions that enables them to also use Internet for company-internal 
communication. Internet has become their site-to-site interconnecting medium 

b) Broadband Internet access. In parallel with the above, more and more 
broadband access solutions are rolled out by different network access providers 

wZtZ 7uT 10 UPgmde aCC6SS t0 10160161 from a ***** dial-up 
PSTN/ISDN (Public Switched Telephone Network/Integrated Services Digital 

Network) access solution to a broadband solution, e.g. ADSL (Asymmetric Digital 
Subscriber Line), Cable or Ethernet, with direct access to Internet. Apart from the 
obvious broadband benefits, the network access user is also able to always be 
connected to the Internet. 

The common name for most of the network solutions that interconnects 
multiple sites over Internet is "virtual private networks" (VPN). VPNs can be 
implemented in numerous ways, this is well explained in e.g. the IETF by B 
Gleeson et. al, "A Framework for IP Based Virtual private Networks" RFC 2764 
February 2000, where IP stands for Internet Protocol. A VPN is a private network 
ftat is configured within a public network. For years, common carriers have built 
VHMs that appear as private national or international networks to the customer but 
Physically share backbone trunks with other customers. VPNs enjoy the security of a 
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private network via access control and encryption, while taking advantage of the 
economies of scale and built-in management facilities of large public networks. 
Today, there is tremendous interest in VPNs over the Internet, especially due to the 
constant threat of hacker attacks. The VPN adds that extra layer of security, and a 
5 huge growth in VPN use is expected. In general, the different VPN solutions can be 
categorized into two main groups; customer premises equipment (CPE) based 
solutions or network based solutions. 

The Internet is a public data network based on network paradigms such as 
equal and best effort traffic treatment. All traffic crossing the Internet is public and 

10 insecure resulting in a number of problems that need to be solved, e.g. end-to-end 
security communication between enterprise sites. Some problems have solutions 
supported by several VPN system vendors, such as encrypted IP tunnelling between 
end-users using the IPSec architecture described by S. Kent and R. Atkinson in 
"Security Architecture for the Internet Protocol", RFC 2401, 

15 November 1998, or stand-alone firewall solutions, desktop software VPN clients, 
e.g. Microsoft® VPN, etc. A PC that is connected to the Internet can, not easily but 
it is possible, be used as a transit node by a hacker, e.g. the hacker could use a 
Trojan horse program to get inside the PC. Well inside, the Trojan horse program 
may be adapted to release application software that will act as some authenticated 

20 software installed by the owner of the PC. It is very difficult for layer- 2 and 3 
firmware/software to detect this kind of malicious applications. Therefore, it is 
recommendable to have VPN control and management software and firmware 
functions and end-user applications, such as service login software, "authenticated" 
software applications that in some way uses the network infrastructure provided by 

25 the VPN service, separated on different hardware platforms. What generally should 
be avoided, is having PC clients that are responsible for configuring the actual VPN 
setup, i.e. having access to the lookup-table for other VPN members public IP 
addresses, having access to information on how to authenticate, perform integrity 
check and encrypt traffic aimed for the VPN etc. 

30 

Summary of the invention 

According to a first aspect of the invention, a system is provided for 
establishment of a virtual private network connection, comprising an end user client 
device and a VPN access server communicatively connected to the end user client 
35 via the Internet. The system is characterised in that it includes a standalone VPN 
client device physically interconnecting the end user client with the Internet, said 
VPN client comprising monitoring means for monitoring all traffic between the end 
user client and the VPN server. Preferably said monitoring means are devised to 
detect when a handshake agreement is established between the end user client and 
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the VPN server. 

Said VPN client comprises, in one embodiment, session overtaking means 
devised to overtake a VPN setup session for said end user client upon detection of 
said handshake agreement. Preferably the end unit client side of the VPN client is 
5 defined as a secure domain, and the Internet and server side of the VPN client is 
defined as an insecure domain, said VPN client being devised only to accept a 
request for a VPN session setup when initialised from said secure domain. 

In one embodiment said monitoring means are devised to determine said 
handshake agreement for the VPN setup session as completed upon detecting that 
10 said server acknowledges a VPN setup request that has been initialised by said end 
user chent. Said VPN client may be devised to request, upon detection of a 
completed handshake, said server to distribute VPN configuring data relevant for 
the inclusion of said end user client into said virtual private network. 

In one embodiment said VPN client is devised to undertake a proxy roll 
15 comprising means for acting as a VPN server proxy towards the end user client, and 
means for acting as an end user client proxy towards the VPN server. 

According to a second aspect, the present invention provides a method for 
establishing a connection for comprising an end user client device to a virtual 
private network controlled by a VPN access server communicatively connected to 
20 the end user client via the Internet, comprising the steps of providing a standalone 
VPN client device physically interconnecting the end user client with the Internet 
and momtoring all traffic between the end user client and the VPN server by means 
of monitoring means in said VPN client. Preferably said monitoring means detects 
when a handshake agreement is established between the end user client and the 
25 VPN server, wherein said VPN client overtakes a VPN setup session for said end 
user client upon detection of said handshake agreement. 

In one embodiment the end unit client side of the VPN client is defined as a 
secure domain, and the Internet and server side of the VPN client is defined as an 
insecure domain, said VPN client only accepting a request for a VPN session setup 
30 when initialised from said secure domain. 

Preferably said monitoring means determine said handshake agreement for the 
VPN setup session as completed upon detecting that said server acknowledges a 
VPN setup request that has been initialised by said end user client. In one 
embodiment said VPN client requests, upon detection of a completed handshake 
35 said server to distribute VPN configuring data relevant for the inclusion of said end 
user client into said virtual private network. In one embodiment said VPN client 
undertakes a proxy roll, acting as a VPN server proxy towards the end user client, 
and actmg as an end user client proxy towards the VPN server. 
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Brief description of the drawings 

Preferred embodiments of the invention are described below with references 
being made to the drawings, on which 
5 Fig. 1 illustrates the system overview according to an embodiment of the 
present invention; 

Fig. 2 illustrates traffic monitoring and session overtaking according to an 
embodiment of the present invention; and 

Fig. 3 illustrates an emulated LAN on top of a global IP network, according to 
10 an embodiment of the invention. 

Detailed description of preferred embodiments 

According to one aspect, the system according to the present invention is based 
on a standard IP network like the public Internet. The system comprises multiple 

15 VPN clients and at least one server. One server can be a distributed cluster of 
physical boxes. The VPN clients could be implemented as drivers on the client 
computer but are for security reasons preferably implemented in a stand alone 
hardware box. A purpose of this mechanism is to establish dynamic and secure 
Virtual Local area Networks between some or all of the clients. A virtual network is 

20 created by establishing connection groups in a VPN server. The server has a service 
device for keeping track of connected machines and mapping them to IP addresses. 
In one embodiment this is obtained using ARP (Address Resolution Protocol), an IP 
protocol used to obtain a node's physical address. A client station sends an ARP 
request to the VPN server with the VPN internal IP address of the target node it 

25 wishes to communicate with, and the VPN server responds by sending back the 
external DP address so that packets can be transmitted. ARP returns the layer-2 
address for a layer-3 address. This mechanism also handles distribution of public 
keys to form complete security associations. For handling broadcasts an emulated 
broadcast service is implemented in the server, preferably using an IP multicast 

30 group or as a separate broadcast service. Data sent directly from one machine in the 
virtual network to another is tunnelled over IP directly to the IP address of the 
receiving client. The mechanism includes both the case where data packets are 
tunnelled directly over IP and when an layer-2 media such as Ethernet is bridged 
onto the IP network. 

35 Fig. 3 illustrates an embodiment of the system according to the present 
mechanism, wherein a network 4 comprises five nodes; four VPN clients 31-34 
with global addresses CI - C4, and a server S. All of these are connected to and 
have a valid address in the physical network 4. These nodes are interconnected 
using standard Internet routing procedures, but the clients 31 - 34 are not on the 
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same LAN. On top of this network infrastructure, clients 31, 32 and 33 form a 
virtual network 30 with local addresses Dl, D2 and D3. In the illustrated case the 
clients in this VPN appear to be on the same local area network. The reason for this 
is the broadcast service, i.e. the service device, which delivers all packets for the 
5 local broadcast domain to all machines on the VPN 30. Thus service discovery 
mechanisms or layer-2 ARP operate transparently on top of the virtual network 
When client 31 on the VPN wants to transmit a packet directly to client 32 the 
client-software requests the physical address C2 from server S, based upon the local 
address D2, and possible security keys required for talking to D2 from S. Dl is then 
10 able to transmit the packet in a secure tunnel directly to D2 without passing the 
server S. 

The above provides an effective and user friendly mechanism for establishing 
Virtual Private Networks over generic IP connections. Broadcast services and 
service discovery protocols that normally require a direct layer-2 interconnection 
15 may work independently of the actual network structure. It also provides the 
possibilities of distributed network broadcast handling, where rules and 
configuration options may be cached in the end nodes of the network instead of in a 
centralised server. The described mechanism is unique in that it presents a complete 
distributed emulated LAN on top of an IP network where access and attributes such 
20 as security associations are completely controlled by a server. Most current solutions 
uses ; static tunnels. Either permanent connections are set up between the members of 
tfie VPN or tunnel servers which basically works as modem pools only you "dial" an 
IP number. This means that all traffic no matter it's final destination goes through 
this one box. In particular traffic going to sites in the VLAN (Virtual LAN) other 
25 than that of the VLAN server comes in through the server access and turns The 
broadcast service allows service discovery protocols designed for local networks to 
function on the VPN while the ARP mechanism allows for dynamic establishment 
ol secure tunnels directly between endpoints. The well known LANE (LAN 
Emulation) standard was focused entirely on ATM (Asynchronous Transfer Mode) 
30 and featured no integrated security handling. Lane introduces, inter aha, the ability 
to connect Ethernet and Token Ring networks together via ATM. LANE makes the 
process transparent, requiring no modification to Ethernet and Token Ring stations. 
LANE allows common protocols, such as IP, IPX, AppleTalk and DECnet to ride 

^ 7Z^ A ™ baCkb ° ne - LAN emU,ati ° n haS been demented and verified over 
35 /OTVL However, since the system architecture itself by design avoids sending all 

data through the server, the bottleneck problem with overloaded server links is 

completely avoided. 

In general, the present invention describes a decision scheme for a third-party 
overtaking of a client role in a two-party communication session. Turning to Fig. 1, 



WSDOCID: <WO___O30O3660A, J_> 



WO 03/003660 PCT/SE01/01472 

6 

the system processes in the illustrated embodiment of the present invention 
comprises end user clients located at the end user premises equipment 1, a central 
VPN system server 2, and network edge located VPN system clients 3. Full lines 
indicate physical communication lines, whereas arrows indicate communicating 
5 ends, without specifying which route the communication takes between those 
communicating ends. 

The end user client process 1 preferably resides in a PC, the VPN client 3 
process preferably resides within a standalone hardware unit, and the VPN server 
process 2 preferably resides within any kind of server hardware unit, such as an 

10 IBM® server. By process is here meant the functionality for the particular client or 
server, as described herein. The VPN server 2 and the VPN client 3 are parts of a 
VPN system that provides the end user client 1 with access to required VPNs. The 
end user client 1 hardware is physically connected via a communication line 1 1 to 
the VPN client 3 hardware. The VPN client 3 hardware is physically connected to a 

15 layer-2 termination that enables the VPN client 3 to access Internet over a 
communication line 12. The layer-2 protocol is preferably Ethernet but could 
practically be any known layer-2 protocol used for the encapsulation and transport 
of IP (Internet Protocol) packets between IP nodes. The VPN server 2 is connected 
to the Internet via a communication line 13 in the same way as the VPN client 3. 

20 According to an embodiment of the invention the end user client 1 initiates a 

communication session with the VPN server 2 in order to acquire access to a virtual 
private network. During the initialisation phase, the VPN server 2 authenticates and 
authorises the end user client 1 as a registered user of VPN services that are 
provided by the VPN server 2. The VPN client 3 is passive in that it does not initiate 

25 any new information elements during the initialisation phase. The VPN client 3 also 
monitors 22 the communication 21 between the end user client 1 and the VPN 
server 2. 

When the initialisation phase between the end user client 1 and the VPN server 
2 is finished, and when information has been exchanged, regarding the particular 

30 VPN that the end user client requests access to, then the VPN client 3 becomes 
active and takes over the communication session between the end user client 1 and 
the VPN client 3. The VPN client 3 now requests, if it is necessary because the VPN 
information can already be cached by the VPN client 3, VPN configuration data 
from the VPN server 2. The VPN client 3 uses the configuration data to configure 

35 necessary VPN access parameters such as traffic classification parameters, 
performance assurance parameters, or firewall parameters such as encryption, 
authentication, filtering parameters, etc. 

The end user client 1 is allowed to use different VPN servers 2 but cannot have 
simultaneous access to more than one VPN server 2. The VPN client 3 detects when 
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an end user eta,, , uies ,„ access . certain server 2 . At ^ 
2 . co^red „ until ^ e „ d user ^ , ^ a 

2 and also have been authenticated by the VPN server 2 

5 in Fi^T m ^r g , a " d SeSSi ° n ° Vertaldng SCenari0S « O**"" — i« detail 
sTde ^^^"^^^^^.wWchisnteendusercUent 1 
stde and one dtstrusted domain, the Internet domain. From the VPN client's 3 point 

of v.ew, me VPN server 2 ia therefore located in me distrusted domain Si^e tuTta 
and o utg „m g ff ^ Mmm ^ ^ ^ ^ ^ *U m- 

hardware the VPN chent 3 is able to monitor the communication 2! bZe „ th 
10 en user chen, 1 and the VPN server 2. This is tnte if, and only if, the TZ^ts 

Z Inn 1 ^ ^ °" ' hat " hySiCaUy ta — * 

15 ^ ? ? ^ C ' iem ' a " d differenl ™ "» 2 <o whom me 

15 end user chent 1 are registered as user. "nomine 

The VPN client 3 identifies when the end user chent 1 starts to establish 
«M wtth a VPN server 2. The VPN chen, 3 beats the end user che u de as a 

ptL 2, ttw t W h "™ 2 * 3 *« - ™e session eatabUahmen. 
Phaae21 between the end uaer client 1 and the VPN server 2 could be done in 
20 numerous ways, e.g. by a traditional challenge/response handshaking sequence The 
—■cation 2, is primarily mean, » be done by web based cbents Z TZ 

t^TZ e " V ~' S °' Uti0,,S *" te ^*in g 

chen, 3 takes over the communication session. The handshaking is considered 
25 ttnshed when the VPN server 2 has authenticated and authorised the Z ^ chent 
1 nd acknowledged the end user client 1 as aconfumed user. The VPNclel, 3 

VPN server 2. Towards the end user client 1, the VPN chen, 3 win ac, as a VPN 
server proxy, and towards the VPN server 2 aa an end user client proxy The end 

souree^TJa ^ * W 3 " C ° nSidering *• WN Se ™ 
35 T ^ P commumcaUon se «ions 23 win, the VPN server 2 that enables 

35 the end user client 1 ,o be included as members in the requested VPN 

In one embodimen, the invention is implemented in a service provisionine 

ysten, where parts of the service functionality are diatiibuted to syste" 

ZZZZZT™: adVa " ,a8eS °' " he PreSMt ■»-*- is that 

any hacker tntrusions via an end user PC 1 are avoided by having critical 
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software/firmware for control and management of VPN configuration data separated 
on standalone hardware 3. Another advantage is the automated overtaking of 
certified sessions. Another benefit is the plug-and-play behavior for virtual services 
over Internet, which is made available through the invention. The teachings of the 
5 present invention thus differs from prior art technology, since earlier solutions to the 
problem have either been centralised server solutions, such as PSTN/ISDN modem- 
pool solutions, server centralised IP Sec tunnelling etc, or distributed solutions, 
which are only valid within one network operator intra-domain or within federated 
network operator domains. These solutions are generally referred to as network 
10 based VPN systems. The present invention will function independently of whether 
or not the different VPN client users access the same network operator domain or a 
federated network domain or have access to totally independent network operator 
domains. 
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Claims 

1. System for establishment of a virtual private network connection, comprising an 
end user client (1) device and a VPN access server (2) communicatively connecL 
5 to the end user chent via the Internet (4), characterised in a standalone VPN client 
h vom interCOnnectin S ai.12.13) the end user client with the Internet 

said VPN chent comprising monitoring means for monitoring (22) all traffic (21) ' 
between the end user client and the VPN server. 

10 2 The system as recited in claim 1, wherein said monitoring means are devised to 
detect when a handshake agreement is established between the end user client and 
the VPN server. 

3. The system as recited in claim 2, wherein said VPN client comprises session 
15 overtakn, g meaas, devised ,„ overiake a VPN setnp session for said end user ctien, 
upon detection of said handshake agreement. 

4 The system as recited in claim 3, wherein the end unit client side of the VPN 
chent is defined as a secure domain, and the Internet and server side of the VPN 
20 chent is defined as an insecure domain, said VPN chent being devised only to 

accept a request for a VPN session setup when initialised from said secure domain. 

5. The system as recited in claim 2, wherein said monitoring means are devised to 

25 agreement f ° r ±e VPN S6tU P Session 38 dieted upon 

25 detecting that said server acknowledges a VPN setup request that has been 
initialised by said end user client. 

6. The system as recited in claim 5, wherein said VPN client is devised to request 
upon detection of a completed handshake, said server to distribute VPN configuring 

30 data relevant for the inclusion of said end user chent into said virtual private 
network. F 

7. The system as recited in claim 1, wherein said VPN cUent is devised to 

35 I H * T y r ° U ' C ° mPriSing ^ f ° r aCtU>8 " 3 WN «"* '"wards 
35 foe end user cheat, and means for acting as an end user client proxy towards the 
VPN server. 

8. Method for establishing a connection for comprising an end user chent (1) 
device to a virtual private network controlled by a VPN access server (2) 
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communicatively connected to the end user client via the Internet (4), comprising 
the steps of: 

- providing a standalone VPN client (3) device physically interconnecting (1 1,12,13) 
the end user client with the Internet; 
5 - monitoring (22) all traffic (21) between the end user client and the VPN server by 
means of monitoring means in said VPN client. 

9. The method as recited in claim 8, wherein said monitoring means detects when a 
handshake agreement is established between the end user client and the VPN server. 

10 

10. The method as recited in claim 9, wherein said VPN client overtakes a VPN 
setup session for said end user client upon detection of said handshake agreement. 

1 1 . The method as recited in claim 10, wherein the end unit client side of the VPN 
15 client is defined as a secure domain, and the Internet and server side of the VPN 

client is defined as an insecure domain, said VPN client only accepting a request for 
a VPN session setup when initialised from said secure domain. 

12. The method as recited in claim 9, wherein said monitoring means determine said 
20 handshake agreement for the VPN setup session as completed upon detecting that 

said server acknowledges a VPN setup request that has been initialised by said end 
user client. 

13. The method as recited in claim 12, wherein said VPN client requests, upon 

25 detection of a completed handshake, said server to distribute VPN configuring data 
relevant for the inclusion of said end user client into said virtual private network. 

14. The method as recited in claim 1, wherein said VPN client undertakes a proxy 
roll, acting as a VPN server proxy towards the end user client, and acting as an end 

30 user client proxy towards the VPN server. 
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